Search for: All records

Large Language Models (LLMs) are often augmented with external contexts, such as those used in retrieval-augmented generation (RAG). However, these contexts can be inaccurate or intentionally misleading, leading to conflicts with the model’s internal knowledge. We argue that robust LLMs should demonstrate situated faithfulness, dynamically calibrating their trust in external information based on their confidence in the internal knowledge and the external context to resolve knowledge conflicts. To benchmark this capability, we evaluate LLMs across several QA datasets, including a newly created dataset featuring in-the-wild incorrect contexts sourced from Reddit posts. We show that when provided with both correct and incorrect contexts, both open-source and proprietary models tend to overly rely on external information, regardless of its factual accuracy. To enhance situated faithfulness, we propose two approaches: Self-Guided Confidence Reasoning (SCR) and Rule-Based Confidence Reasoning (RCR). SCR enables models to self-access the confidence of external information relative to their own internal knowledge to produce the most accurate answer. RCR, in contrast, extracts explicit confidence signals from the LLM and determines the final answer using predefined rules. Our results show that for LLMs with strong reasoning capabilities, such as GPT-4o and GPT-4o mini, SCR outperforms RCR, achieving improvements of up to 24.2% over a direct input augmentation baseline. Conversely, for a smaller model like Llama-3-8B, RCR outperforms SCR. Fine-tuning SCR with our proposed Confidence Reasoning Direct Preference Optimization (CR-DPO) method improves performance on both seen and unseen datasets, yielding an average improvement of 8.9% on Llama-3-8B. In addition to quantitative results, we offer insights into the relative strengths of SCR and RCR. Our findings highlight promising avenues for improving situated faithfulness in LLMs. 

Dynamic bonds are a powerful approach to tailor the mechanical properties of elastomers and introduce shape-memory, self-healing, and recyclability. Among the library of dynamic crosslinks, electrostatic interactions among oppositely charged ions have been shown to enable tough and resilient elastomers and hydrogels. In this work, we investigate the mechanical properties of ionically crosslinked ethyl acrylate-based elastomers assembled from oppositely charged copolymers. Using both infrared and Raman spectroscopy, we confirm that ionic interactions are established among polymer chains. We find that the glass transition temperature of the complex is in between the two individual copolymers, while the complex demonstrates higher stiffness and more recovery, indicating that ionic bonds can strengthen and enhance recovery of these elastomers. We compare cycles to increasing strain levels at different strain rates, and hypothesize that at fast strain rates ionic bonds dynamically break and reform while entanglements do not have time to slip, and at slow strain rates ionic interactions are disrupted and these entanglements slip significantly. Further, we show that a higher ionic to neutral monomer ratio can increase the stiffness, but its effect on recovery is minimal. Finally, taking advantage of the versatility of acrylates, ethyl acrylate is replaced with the more hydrophilic 2-hydroxyethyl acrylate, and the latter is shown to exhibit better recovery and self-healing at a cost of stiffness and strength. The design principles uncovered for these easy-to-manufacture polyelectrolyte complex-inspired bulk materials can be broadly applied to tailor elastomer stiffness, strength, inelastic recovery, and self-healing for various applications.